Video signal processing device and its method

ABSTRACT

Since the corner detection means (7, 9) to detect each position of the blue board area from video signal, the conversion address generation means (11) to generate the conversion address based on the position information detected and the position information showing each corner position of the image area of video signal to be inserted and the image conversion means (16) to form conversion source video signal based on the conversion address are provided and the source video signal is to be inserted to the video signal, the operator&#39;s operation adjustment as the conventional device becomes unnecessary when inserting the source video signal into the prescribed frame of the video signal, and the operation of the operator can be further decreased. Thus, a video signal processing device capable of further decreasing the operator&#39;s operation can be realized.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a video signal processing device and avideo signal processing method, and is suitably applied to a videosignal processing device, for example, a newscaster who is reading aloudthe news in the news studio is photographed and the video signal isgenerated and by inserting the other video signal into that videosignal, the image of the other video signal is inserted into theprescribed frame in the background of the newscaster.

2. Description of the Related Art

Heretofore, in the news program production, by inserting the video imagematching to the content of the news being read by the newscaster to theprescribed frame in the background of the newscaster, the imagesmatching to the content of that news can be provided to the audience.Thus, the audience can comprehend the content of that news being readaloud by the newscaster more in detail according to the video image tobe shown in the background of the newscaster.

Heretofore, these video signal insertion processings have been conductedaccording to the method to be discussed in the following.

Firstly, a source video signal to show the video image to be inserted tothe prescribed frame in the background of the newscaster (heretoforereferred to as a source video image) is formed (or reproduced from theVTR) and this is supplied to one end of an input terminal of a mixerthrough a video effector. At the same time, by photographing thenewscaster in the studio, studio video signal is obtained and this issupplied to the other end of the input terminal of the mixer. This mixercombines the source video signal entered in one end of the inputterminal and the studio video signal entered in the other end of theinput terminal and supplies the resultant composite video signal to amonitor.

An operator supplies parameter for scaling up and down, moving androtation to a video effector through an input device such as a trackballin order that the outer shape of the source video image fits to theprescribed frame in the background of the newscaster by observing thecomposite image displayed on the monitor. The video effector conductsthe processing of scaling up and down, moving and rotation to the sourcevideo image based on the parameter supplied, and supplies the sourcevideo signal processed and the key signal showing the shape of theprocessed source video image to the mixer. The mixer inserts the sourcevideo signal into the position shown by this key signal from amongstudio video signal. The resultant composite video signal is supplied tothe monitor as described above and displayed. The operator, repeatingthe parameter of scaling up and down, moving and rotation till theexternal shape of the source video image fits to the prescribed frame inthe background of the newscaster by observing the composite image to bedisplayed, supplies to the video effector.

In the case of conventional device, by successively repeating theseprocessings, source video image has been inserted to the prescribedframe in the background of the newscaster.

However, in the conventional insertion processing, the operator has tomanually input the parameter necessary for the conversion processing,such as scaling up and down, moving and rotation of the source videoimage in utilizing the input device such as trackball until the shape ofsource video image fits to the prescribed frame in the background of thenewscaster, and thus, it takes comparatively long time before the shapeof source video image completely fits to the prescribed frame in thebackground of the newscaster and since the operator must conduct aninput operation of the parameter during that period, it causes a problemthat the operation becomes complicated.

As a method to solve this problem, there is a method disclosed in theU.S. Patent (U.S. Pat. Nos. 4,951,040 and 5,107,252). In this method,the operator inputs at least 4 points of corner position showing theshape after the source video image is converted by using the input meanssuch as touch tablet and stylus. And the image conversion means convertssaid source video image in order that the corners of the source videoimage fit to 4 corners specified on the basis of the address signalshowing 4 points of corner position entered. According to this method,the source video image can be inserted into the prescribed frame withina comparatively short period of time as compared with the methoddescribed earlier and the operator's operation can be decreased.

However, according to this method in the case of inserting the sourcevideo image into the prescribed frame in the background of thenewscaster, the operator must input corner positions by manuallyoperating the input means such as touch tablet and stylus in order thateach corner of the source video image fits to each corner of theprescribed frame in the background of the newscaster, and it is stillinsufficient on the point to simplify the operation of the operator.

SUMMARY OF THE INVENTION

In view of the foregoing, an object of the present invention is toprovide a video signal processing device and a video signal processingmethod capable of further decreasing the operation task of the operator.

To obviate such problems according to the present invention, we providecorner detection means for detecting the blue board area from the videosignal obtained by photographing an object with the blue board in thebackground of said object to be photographed and detecting each cornerposition of said area respectively; source specification means forspecifying each corner of the image area to be inserted to the blueboard area of the video signal from among the source video signal; aposition information to show each corner position of the blue board areadetected by the corner detection means; conversion address generationmeans for generating the conversion address to change the image area tothe same shape as the blue board area upon image transforming the imagearea based on the position information to show each corner position ofthe image area specified by the source corner specification means; imageconversion means for forming the converted source video signal in whichimage area is transformed to the same shape as the blue board area bywriting the source video signal in the memory and reading out the sourcevideo signal written in the memory on the basis of the conversionaddress generated by the conversion address generation means; and signalmixing means for forming composite video signal of which image area ofthe source video signal is inserted to the blue board area in the videosignal by mixing the conversion source video signal and the videosignal.

Accordingly, since each corner position of the blue board area isdetected from the video signal respectively, and the conversion addressis generated on the basis of the position information to show theposition detected and the position information to show each cornerposition of image area of the video signal to be inserted and theconversion source video signal will be formed on the basis of thatconversion address, in the case of inserting the source video signalinto the prescribed frame of the video signal, image area can beautomatically inserted into the blue board area even if the operatorwould not enter the parameter to show the condition after conversion asbefore, the operation of the operator can be further decreased ascompared with the conventional device.

The nature, principle and utility of the invention will become moreapparent from the following detailed description when read inconjunction with the accompanying in which like parts are designated bylike reference numerals or characters.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 is a block diagram showing the general construction of a videosignal processing device according to one embodiment of the presentinvention;

FIGS. 2A and 2B are brief linear diagrams showing images of the targetkey signal keyT and the studio video signal V_(BK) ;

FIGS. 3A and 3B are brief linear diagrams illustrating the rangespecification of the selected image area;

FIG. 4 is a brief linear diagram showing an image of the source videosignal V_(out) ;

FIG. 5 is a brief linear diagram showing an image of the self key signalS5;

FIG. 6 is a brief linear diagram showing an image of the self key signalK_(out) ;

FIG. 7 is a brief linear diagram showing an image of the composite videosignal V_(mix) when the whole effective image area EFGH is specified asthe selected image area IJKL;

FIG. 8 is a brief linear diagram showing an image of the composite videosignal V_(mix) when a part of effective image area EFGH is specified asthe selected image area IJKL;

FIG. 9 is a block diagram showing the construction of an imagetransducer;

FIGS. 10A and 10B are brief linear diagrams illustrating the principleof the 3-D image conversion processing;

FIG. 11 is a brief linear diagram illustrating the correlation ofposition vectors between a memory and a monitor screen;

FIGS. 12A to 12C are brief linear diagrams showing images of each videosignal during the process of inserting the source video signal V_(in) tothe studio video signal V_(BK) ;

FIGS. 13A to 13C are brief linear diagrams showing images of each videosignal during the process of inserting the source video signal V_(in) tothe studio video signal V_(BK) ;

FIG. 14 is a brief linear diagram illustrating the case of inserting theselected image area IJKL to the quadrangle ABCD without rotating;

FIG. 15 is a brief linear diagram illustrating the case of fitting theselected image area IJKL to the quadrangle turning 90-degree in theclockwise;

FIG. 16 is a brief linear diagram illustrating the case of fitting theselected image area IJKL into the quadrangle turning 180-degree in theclockwise;

FIG. 17 is a brief linear diagram illustrating the case of fitting theselected image area IJKL into the quadrangle turning 270-degree in theclockwise;

FIGS. 18A to 18C are brief linear diagrams illustrating the case ofspecifying an optional shape as the selected image area IJKL; and

FIG. 19 is a block diagram showing the construction of a video signalprocessing device according to the other embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENT

Preferred embodiment of the present invention will be described withreference to the accompanying drawings:

(1) General Construction

In FIG. 1, 1 generally shows a video signal processing device accordingto the present invention and a studio video signal will be formed byphotographing a studio 3 with a video camera 2. In this case, anewscaster 4 is set to read the news text at the position adjacent to atable 5 located in the studio 3. Moreover, a quadrangle ABCD blue board6 having blue hue is placed in the depth direction slanted behind thenewscaster 4. This blue board 6 is placed to show the insertion positionof the source video image, and the source video image will be insertedinto this blue board 6 by image mixing to be described later. In thisconnection, in the case of not inserting the source video image, theblue board 6 will be transferred to the position T out of photographicrange of the video camera 2 by moving up and down mechanism of themotor.

At the rear of the newscaster 4 and the blue board 6, there is a wall ofthe studio 3 and the hue of this wall is selected to the hue other thanblue so that the blue board 6 can be easily recognized.

The studio 3 arranged in this manner will be photographed by the digitalsystem video camera 2 having CCD as an image pickup device. In thiscase, the video camera 2 photographs the studio 3 in order that the blueboard 6 and the table 5 stay in the screen placing the newscaster 4 inthe center of the screen. Digital image signal to be put out by thisvideo camera 2 will be supplied to a chroma key device 7 as a studiovideo signal V_(BK) and simultaneously supplied to one side of the inputterminal of a mixer 8.

The chroma key device 7 detects image signal having blue hue from thestudio video signal V_(BK) supplied and outputs the detection result asa target key signal keyT. This target key signal keyT is the digitalsignal of 10 bit, and the signal level of the target key signal keyTshown by this digital signal becomes the level corresponding to the hueof the studio video signal. More specifically, in the area of videosignal having blue hue, the signal level of the target key signal keyTbecomes high, and in the area of video signal having other than bluehue, the signal level of the target key signal keyT becomes low.Accordingly, the shape of the area having "high" signal level agreeswith the shape of the blue board 6, and as shown in FIG. 2A, it becomesa quadrangle ABCD as same as the blue board 6. Also, the area of whichsignal level becomes "high" agrees with the position of the blue board 6in the studio video signal V_(BK). For reference purposes, the studiovideo signal will be shown in FIG. 2B.

Accordingly, the target key signal keyT showing the shape and positionof the blue board 6 will be supplied to the following corner detector 9.

The corner detector 9 receives reference signal level. S_(LEV) enteredby the operator using the volume for reference level provided in aninput device 10 from said input device 10 and compares said referencesignal level S_(LEV) and the signal level of the target key signal keyTsupplied from the chroma key device 7. Then, the corner detector 9,detecting the area of which the signal level of the target key signalkeyT becomes larger than the reference signal level S_(LEV), detects theblue board 6 area, and detecting positions of 4 corners of that area onthe corner display coordinates, generates address signal S1 to show 4corner positions. This address signal S1 is supplied to the followingthree-dimensional conversion address generator 11. Here, the displaycoordinates refers to as coordinates having the horizontal direction ofthe monitor screen to be x axis and the vertical direction to be y axis,and the perpendicular direction to the monitor screen as z axis.

On the other hand, in the case of this video signal processing device 1,the source video signal into be inserted to the prescribed frame in thebackground of the newscaster 4 is formed by two methods which will bedescribed as follows. The first method is a method to form the sourcevideo signal by reproducing the video signal prerecorded on the magnetictape by using an video tape recorder (VTR) 12. The second method is amethod to form the source video signal by photographing an object 14using a video camera 13 having CCD as an image pickup element. Here, thevideo tape recorder 12 and the video camera 13 are both digital systemdevices.

Output terminals of the video tape recorder 12 and the video camera 13are connected to one side of the input terminal of a switch 15 and tothe other side of the input terminal. Thus, as the operator switches theswitch 15, the desired video signal can be selectively obtained from thesource video signal to be formed by two methods. The digital videosignal selected by the switch 15 is supplied to an image converter 16 asa source video signal V_(in).

In this connection, the source video signal V_(in) is an image signal ofcolor image system (e.g., NTSC system) having effective image area EFGHas shown in FIG. 1 even in the case where the source video signal V_(in)is formed by either one of above methods.

At this point, in the video signal processing device 1, of the effectiveimage area EFGH of the source video signal V_(in) thus formed, thedesired image area is specified and this can be inserted to theprescribed frame (ABCD) placed in the background of the newscaster 4.This image area specification will be described in detail in thefollowing. Provided that in the following description, the image areaselected by the operator will be referred to as the selected image areaIJKL.

Firstly, the source video signal V_(in) is supplied to the imageconverter 16 as described above. This source video signal V_(in) is alsosupplied to a monitor 17. The monitor 17 is a device to show the sourcevideo signal V_(in) and displays effective image area EFGH of the sourcevideo signal V_(in) on the monitor screen.

Display control signal from a controller 18 is supplied to the monitor17 and in order that the selected image area IJKL can be visuallyidentified, identification line 17A showing the outer frame of theselected image area IJKL will be displayed on the screen of monitor 17based on this display control signal. At the time when the operatorenters the range specification information in utilizing the rangespecification volume and a keyboard provided in the input device 10,this identification line 17A changes its size. More specifically, whenthe operator enters the range specification information using the rangespecification volume and the keyboard of the input device 10, thecontroller 18 forms display control signal corresponding to the rangespecification information and controls the size of the identificationline 17A by supplying said display control signal to the monitor 17.With this arrangement, the identification line 17A having the sizespecified by the operator will be displayed on the screen of the monitor17. Thus, the operator may operate the range specification volume andthe keyboard of the input device 10 by observing the screen displayed onthe monitor 17 in order that the desired area to be inserted as thesource video image would be surrounded by the identification line 17A.

At this point, the range specification will be described morespecifically. As described above, the operator operates the rangespecification volume and the keyboard of the input device 10 observingthe source video image to be displayed on the monitor 17 and enters therange specification information, i.e., which range of the effectiveimage area EFGH would be selected as the selected image area IJKL. Inthis case, the operator enters the range specification information inthe horizontal direction and the vertical direction making the center ofeffective image area EFGH of the source video signal V_(in) asreference. For example, if the operator enters plus 80 percent and minus80 percent from the center of the effective image area EFGH as the rangespecification information in the horizontal direction and if theoperator enters plus 80 percent and minus 80 percent from the center ofthe effective image area EFGH as the range specification information inthe vertical direction, and an identification line 17A will be displayedon the position as shown in FIG. 3A. More specifically, the verticalline of the identification line 17A is displayed on the position shiftedplus 80 percent and minus 80 percent in the horizontal direction fromthe center of the effective image area EFGH and the horizontal line ofthe identification line 17A is displayed on the position shifted plus 80percent and minus 80 percent in the vertical direction. In this case,the image area surrounded by the identification line 17A thus displayedis specified as the selected image area IJKL.

Furthermore, for example, if the operator enters plus 50 percent andminus 50 percent from the center of the effective image area EFGH as therange specification information in the horizontal direction and plus 80percent and minus 20 percent as the range specification information inthe vertical direction, the identification line 17A will be displayed onthe position as shown in FIG. 3B. More specifically, a vertical line ofthe identification line 17A is displayed on the position shifted plus 50percent and minus 50 percent in the horizontal direction from the centerof the effective image area EFGH and a horizontal line of theidentification line 17A is displayed on the position shifted plus 80percent and minus 20 percent in the vertical direction from the centerof the effective image area EFGH. In this case, the image areasurrounded by the identification line 17A thus displayed is specified asthe selected image area IJKL.

As the range specification information in the horizontal direction, ifplus 100 percent and minus 100 percent are entered from the center ofthe effective image area EFGH, and as the range specificationinformation in the vertical direction, if plus 100 percent and minus 100percent are entered from the center of the effective image area EFGH,the specification line 17A lies on the contour of the effective imagearea EFGH and this means that the effective image area EFGH itself isspecified as the selected image area IJKL. In this connection, since thedefault value of the specification line 17A is set to plus 100 percentand minus 100 percent both in the horizontal and vertical directions,the effective image area EFGH is specified as the selected image areaIJKL if the operator does not operate the range specification volume andthe keyboard of the input device 10.

At this point, returning to FIG. 1, an explanation on this FIG. 1 willbe continued. When the operator completes the specification of theselected image area IJKL, the controller 18 detects 4 corner positionsof the selected image area IJKL specified based on the rangespecification information from the input device 10 and generates anaddress signal S2 to show the position on the display coordinates of 4corners. If the operator does not conduct the range specification asdescribed above, it generates the address signal S2 showing positions of4 corners of the effective image area EFGH, that is the default value.This address signal S2 will be supplied to a three dimensionalconversion address generator 11 and a self key generator 19respectively. As it is clear from this explanation, the controller 18comprises the source corner specification circuit to specify each cornerposition of image area in which studio video signal V_(BK) is insertedto the three-dimensional conversion address generator 11 to be describedin the following paragraphs.

As described above, the address signal S1 showing 4 corner positions ofthe quadrangle ABCD showing the blue board 6 supplied from the cornerdetector 9 and the address signal S2 showing 4 corner positions of theselected image area IJKL supplied from the controller 18 are supplied tothe three-dimensional conversion address generator 11. Furthermore,screen address signal S3 formed by the screen address generator 22 issupplied into this three-dimensional conversion address generator 11.This screen address signal S3 is a signal showing the address on themonitor screen of the monitor 21 to be described later. The screenaddress generator 22 is equipped with a reference clock generatorcorresponding to pixel frequency inside, and on the basis of thereference clock generated by said reference clock generator, generatesscreen address on the monitor 21 in a raster scanning order and outputsthis as the screen address signal S3.

The three-dimensional address generator 1 calculates the conversionaddress which converts the selected image area IJKL to the same shape asthe quadrangle ABCD based on the address signal S1 showing 4 cornerpositions of the quadrangle ABCD showing the blue board 6 supplied fromthe corner detector 9 and the address signal S2 showing 4 cornerpositions of the selected image area IJKL supplied from the controller18.

More specifically, the three-dimensional address generator 11 calculatesthe transformation matrix in order that the selected image area IJKLwhich is applied the natural perspective conversion processing, isinserted to the quadrangle ABCD based on the address signal S1 to show 4corner positions of the quadrangle ABCD and the address signal S2 toshow the 4 corner positions of the selected image area IJKL. Then, thethree-dimensional address generator 11 calculates the inverse matrix ofthat transformation matrix and calculates conversion address bysuccessively multiplying said inverse matrix to the screen addressobtained by the screen address signal S3. This conversion address willbe supplied to the image converter 16 as a conversion address signal S4.

The image converter 16 is comprised of field memory and writes thesource video signal V_(in) to be entered in the field memory. Moreover,the image converter 16, by reading the source video signal V_(in) fromthe position in the field memory to be specified by the conversionaddress signal S4 supplied from the three-dimensional conversion addressgenerator 11, forms source video signal V_(out) in which its selectedimage area IJKL as a source video image is converted to the quadrangleABCD having the same shape as the blue board 6. In this connection,since the conversion address is formed on the basis of the screenaddress formed in a raster scanning order, there are cases where theconversion address shows the position not existing in the field memory.In that case, the image converter 11 does not conduct the read operationof the source video signal V_(in).

With this arrangement, the source video signal V_(out) thus formed, asshown in FIG. 4, in which the selected image area IJKL is the sourcevideo image, is transformed to the same shape as the quadrangle ABCDshowing the blue board 6, and is a video signal coordinate transformedto the position of that quadrangle ABCD. As it is clear from this FIG.4, the relationship between the selected image area EFGH and thequadrangle ABCD is so arranged that corners E, F, G and H of theselected image area correspond to corners A, B, C and D of thequadrangle respectively.

The source video signal V_(out) thus formed will be supplied to theother side of the input terminal of the mixer 8.

The self key generator 19 generates self key signal S5 in which an areahaving the signal level "1" becomes the same shape as the selected imagearea IJKL and outputs this to a key signal converter 20 on the basis ofthe address signal S2 showing 4 corner positions of the selected imagearea IJKL supplied from the controller 18. As it is clear from this FIG.5, the size of whole area shown by the self key signal S5 corresponds tothe effective image area EFGH.

Basically, the key signal converter 20 has the construction similar tothat of the image converter 16, and sequentially writes the self keysignal S5 to be entered in the field memory. Moreover, the key signalconverter 20, by reading out the self key signal S5 from the position inthe field memory to be specified by the conversion address signal S4supplied from the three-dimensional conversion address generator 11,forms self key signal K_(out) in which the area with the signal level"1" is converted to the quadrangle ABCD having the same shape as theblue board 6. Also in the case of this converter 11, if the conversionaddress shows the position not existing in the field memory, the readoperation of the self key signal S5 would not be conducted.

As shown in FIG. 6, thus formed self key signal K_(out) is the signal inwhich the area with the signal level "1" is converted to the same shapeas the quadrangle ABCD showing the blue board 6, and the coordinatetransformed to the position of that quadrangle ABCD.

Thus formed self key signal K_(out) will be supplied to the key inputterminal of the following mixer 8.

The mixer 8 combines the source video signal V_(out) supplied from theimage converter 16 and the studio video signal V_(BK) supplied from thevideo camera 2 on the basis of the self key signal K_(out) supplied tothe key signal input terminal. More specifically, the mixer 8 outputsthe studio video signal V_(BK) supplied from the video camera 2 at thetime when the signal level of the self key signal K_(out) is "0", andoutputs the source video signal V_(out) supplied from the imageconverter 16 when the signal level of the self key signal K_(out) is"1". Thus, the mixed video signal V_(mix) in which the source videosignal V_(out) is inserted to the blue board 6 area of the studio videosignal V_(BK) is formed. This mixed video signal V_(mix), as well asbeing supplied to the outside broadcasting facilities, is supplied tothe monitor 21.

When the mixed video signal V_(mix) thus formed is displayed on themonitor 21, the mixed video screen in which the source video image IJKLis inserted into the prescribed frame ABCD in the background of thenewscaster 4 will be displayed on this monitor 21.

In this connection, in the case where the whole area of effective imagearea EFGH is specified as the selected image area IJKL, as shown in FIG.7, the mixed video screen in which the whole area of effective imagearea EFGH is inserted to the prescribed frame ABCD in the background ofthe newscaster 4 will be displayed.

Furthermore, in the case where a part of effective image area EFGH(i.e., only flower part) is specified as the selected image area IJKL asshown in FIG. 3B, the mixed video screen in which only specified part isinserted to the prescribed frame ABCD in the background of thenewscaster 4 will be displayed as shown in FIG. 8. As it is clear fromthis FIG. 8, in the case where only a part of the effective image areaEFGH is specified, that specified part will be displayed enlarged.

(2) Construction of Image Converter

In this chapter the construction of the image converter 16 describedabove will be explained more specifically. As shown in FIG. 9, the imageconverter 16 comprises a filter 16A, a memory 16B, an interpolator 16C,a write address generator 16D and a read address generator 16E. Thesource video signal V_(in) supplied from a switch 15 is firstly suppliedto the filter 16A. This filter 16A is to control the occurrence ofaliasing and provides the band control in the horizontal and verticaldirections to the source video signal V_(in) to be entered, and suppliesthe resultant band controlled source video signal V_(in) to the memory16B.

This memory 16B is comprised of three field memories. Of 3 fieldmemories, one is constantly controlled in a writable state, and theremaining two are controlled in readable states. In these cases, thefield memories to be controlled in writable states and readable stateswill be sequentially shifted in a field cycle. For example, at first, ifthe first field memory is in a writable state and the second and thethird field memories are in readable states, at the following fieldtiming, the second field memory is controlled in the writable state andthe third and the first field memories are controlled in the readablestates. Furthermore, at the following field timing, the third fieldmemory is controlled in the writable state and the first and the secondfield memories are controlled in the readable states. Since the writablecondition and readable condition of 3 field memories will be shifted ina field cycle, the conversion processing to be described in thefollowing paragraph can be conducted in real time.

When the source video signal V_(in) is entered, the memory 16Bsequentially writes the source video signal V_(in) in the field memorywhich is in a writable state on the basis of sequential write addresssignal S10 to be supplied from the write address generator 16D. Thewrite address generator 16D has a reference clock generatorcorresponding to the pixel frequency of the source video signal V_(in)inside and generates sequential address signal S10 on the basis of thereference clock generated in that reference clock generator.

Moreover, paralleling this write operation, the memory 16B successivelyreads the source video signal V_(in) from 2 field memories which are ina readable state on the basis of read address signal S11 to be suppliedfrom the read address generator 16E. This source video signal read outwill be supplied to the following interpolator 16C.

At this point, the read address generator 16E forms read address signalS11 on the basis of the conversion address signal S4 supplied from thethree-dimensional conversion address generator 11. In this case, theread address generator 16E takes out an integer part of the conversionaddress to be obtained by the conversion address signal S4 as the readaddress and supplies that read address to the memory 16B as an addresssignal S11. If the read address pulled out is the address not existingin the memory 16B, the read address signal S11 would not be produced andthe read operation stops as described above.

Furthermore, the read address generator 16E takes out a decimal part ofthe conversion address to be obtained by the conversion address signalS4 and on the basis of that decimal part, forms an interpolationcoefficient for use in the interpolator 16C. This interpolationcoefficient will be supplied to the interpolator 16C as an interpolationcoefficient signal S12.

The interpolator 16C performs the interpolation processing on the sourcevideo signal read out from the memory 16B and it provides theinterpolation processing to the source video signal read out based onthe interpolation coefficient to be obtained from the interpolationcoefficient signal S12. The reason that the interpolator 16C is providedhere is as follows: Since the conversion address to be supplied from thethree-dimensional conversion address generator 11 is not always theinteger but it contains the decimal sometimes. Accordingly, if theconversion address contains the decimal, read operation cannot beconducted since the decimal address does not exist in the memory 16B.Therefore, the conversion address is divided into the integer part andthe decimal part, and in the case where the conversion address containsdecimals, the video data read out by the integer part is interpolationprocessed and the video data corresponding to the decimal part isobtained. Thus, in the case where the conversion address contains thedecimal, the video data corresponding to that conversion address can beobtained.

Thus, by reading out the source video signal from the memory 16Ccorresponding to the integer part of the conversion address andproviding the interpolation processing onto the source video signal readout corresponding to the decimal part of the conversion address, asshown in FIG. 4, the source video image part is transformed to the sameshape as the quadrangle ABCD showing the blue board 6 and the coordinateconverted source video signal V_(out) is formed on the position of thatquadrangle ABCD. This source video signal V_(out) will be supplied tothe mixer 8 as described above.

In this connection, we have described so far that a set of filter 16A,memory 16B and interpolator 16C is provided. However, in practice, 2sets of filter 16A, memory 16B and interpolator 16C are providedcorresponding to the brightness signal and color difference signal ofthe source video signal V_(in). More specifically, in this imageconverter 16, the conversion processing of brightness signal of thesource video signal V_(in) is conducted in one of 2 sets and theconversion processing of color difference signal of the source videosignal V_(in) will be conducted in the other set.

(3) Conversion Address Generating Method of 3-D Conversion AddressGenerator

In this chapter the conversion address generation method in athree-dimensional conversion address generator 11 will be described. Inthe case of inserting the source video signal V_(in) into the quadrangleABCD shown by the blue board 6, the source video signal V_(in) is mappedin the three-dimensional space and it should be further inserted to thequadrangle ABCD after being focused onto the monitor screen making theoperator's visional point as a datum point. The reason is that the blueboard 6 exists in the three-dimensional space and the quadrangle ABCD isan image which a blue board 6 existing in the three-dimensional space isfocused on the monitor screen making the operator's visional point as adatum point. Accordingly, in the three-dimensional conversion addressgenerator 11, the transformation matrix including the image mapping tothe three-dimensional space and throwing it on the two-dimensional planefrom the three-dimensional space should be calculated and by calculatingthe inverse matrix of that transformation matrix, conversion addressshould be formed. More specific explanation will be given on this pointin the following chapters.

(3-1) Definition of Coordinate System

Firstly, the coordinate system of three-dimensional space will bedescribed with reference to FIGS. 10A and 10B. The three-dimensionalcoordinate system being used in this embodiment is defined, as shown inFIGS. 10A and 10B, according to the xyz orthogonal coordinates makingthe center of the monitor screen as an original point, and horizontaldirection of the monitor screen to be x axis, vertical direction of themonitor screen to be y axis, and in the direction perpendicular to themonitor screen to be z axis. In this case, regarding x axis, rightdirection of the monitor screen is taken to be plus direction and leftdirection of the monitor screen is taken to be minus direction, and asto y axis, upper direction of the monitor screen is taken as plusdirection and lower direction of the monitor screen is taken to be minusdirection, and as to z axis, depth direction of the screen is taken tobe plus direction and this side of the screen (i.e., the side where theoperator's visional point exists) as minus direction.

Furthermore, as regards to the x-axis direction in the screen area,virtual coordinate values between -4 and +4 are set, and as to the yaxis direction in the screen area, virtual coordinate values between -3and +3 are set. Of course, virtual coordinate values are set in theoutside of the screen area. Moreover, regarding the operator's visionalpoint PZ, it is virtually set at the point z coordinate on z axis is"-16".

(3-2) Basic Algorithm of 3-dimentional Image Transformation

Next, at this point, a method for forming the video signal in which thesource video signal V_(in) is 3-dimensional image transformationprocessed (i.e., image mapping in the 3-D space and image focusing onthe monitor screen from the 3-D space) will be described.

Firstly, the source video signal V_(in) is stored in the memory 16B inthe image converter 16 as it is without being given anythree-dimensional processing. Since this source video signal V_(in) istwo-dimensional video signal, as shown in FIG. 10A, this is a videosignal V₁ existing at the position M₁ on the monitor screen in thethree-dimensional space.

This source video signal V_(in) should be coordinate transformed to theposition of blue board 6 which exists in the three-dimensional space asdescribed above. Here, as shown in FIG. 10A, supposing that the blueboard 6 exists on the position M₂, slanted about 45-degree with respectto the screen surface in the plus direction of z axis. If the blue board6 exists on such a position M₂, parallel transfer in the plus directionof z axis, and approximately 45-degree rotation processing about y axismust be conducted to the source video signal V_(in). These coordinatetransformation processing can be executed using a three-dimensionaltransformation matrix T₀. More specifically, by multiplying thethree-dimensional transformation matrix T₀ by each pixel of the sourcevideo signal V_(in), video signal V₂ in the three-dimensional space canbe formed.

In general, this 3-D transformation matrix T₀ can be expressed by thefollowing equation: ##EQU1##

The transformation parameters r₁₁ -r₃₃ to be used in thisthree-dimensional transformation matrix T₀ are parameters containingelements to rotate the source video signal V_(in) about the x axis,y-axis and z-axis respectively, to scale up and down the source videosignal V_(in) in the x-axis direction, in the y-axis direction and inthe z-axis direction respectively, and the element to skew the sourcevideo signal V_(in) in the x-axis direction, in the y-axis direction andin the z-axis direction respectively. Moreover, the transformationparameters 1_(x), 1_(y), 1_(z) are the parameters containing elements tomove the source video signal V_(in) parallel in the direction of x-axis,y-axis and z-axis, and the transformation parameter s is the parametercontaining the element to scale up and down the source video signalV_(in) in the respective axis direction of 3-dimension.

In this connection, since this transformation matrix T₀ expresses thecoordinate system of rotation transformation and the coordinate systemof parallel transfer transformation and scaling up and downtransformation in the same one coordinate system, it becomes a 4-row4-column matrix. In general, such coordinate system is called asHomogeneous Coordinate.

Since the video signal V₂ coordinate transformed in thethree-dimensional space using the 3-dimensional transformation matrix isput in the quadrangle ABCD in the studio video signal V_(BK), imagefocus processing on the monitor screen making the operator's visionalpoint as a datum point should be conducted. More specifically, as shownin FIG. 10A, when video signal V₂ on the position M₂ in the3-dimensional space is viewed from the virtual visional point PZ on thez-axis, video signal V₃ to be seen through on the xy plane must beobtained. This image focus processing can be executed by using aperspective transformation matrix P₀. More specifically, by multiplyingthe perspective transformation matrix P₀ by each pixel of the videosignal V₂, the video signal V₂ existing in the 3-dimensional space canbe obtained as the video signal V₃ seen through on the xy plane.

In general, this perspective transformation matrix P₀ can be expressedas follows: ##EQU2##

The parameter P_(z) of this perspective transformation matrix P₀ is aperspective value for applying the perspective law when the video signalV₂ is seen through on the xy plane. Generally, this perspective valueP_(z) is set on "1/16" as the reference value. This means that the valueof z coordinate of the virtual visional point PZ is "-16", and thisperspective value P_(z) can be changed to the desired value by theoperator's setting.

Thus, by conducting the coordinate transformation to the 3-dimensionalspace and the image focus processing on the source video signal V_(in),it becomes possible that the source video signal V_(in) can be fit inthe quadrangle ABCD in the studio video signal V_(in).

The contents of the transformation processing described above may besummarized as follows: The transformation processing is composed of aspatial image transforming step, that is the step till the 3-dimensionalconversion video signal V₂ is obtained from the source video signalV_(in) (V₁) by the 3-dimensional transformation matrix T₀, and aperspective transforming step, that is the step till the perspectiveconversion video signal V₃ is obtained from the 3-dimensional transformvide o signal V₂ by the perspective transformation matrix P₀.Accordingly, the transformation matrix T to obtain the perspectivetransform video signal V₃ from the source video signal V_(in), (V₁) canbe expressed by the following equation by multiplying the 3-dimensionaltransformation matrix T₀ by the perspective transformation matrix P₀.##EQU3##

Accordingly, by multiplying the transformation matrix T₀ shown in thisequation (3) by each pixel of the source video signal V_(in) the sourcevideo signal V_(out) which can be inserted into the studio video signalV_(BK) can be formed.

In this video signal processing device 1, in the case of forming thesource video signal V_(out), the transformation matrix P₀ is notmultiplied by the source video signal V_(in), but in practice, byobtaining the read address on which the image transformation shown bythe transformation matrix T₀ will be applied, the source video signalV_(out) will be formed by reading the source video signal from thememory 16B of the image converter 16 based on that read address.

More specifically, in this video signal processing device 1, the sourcevideo signal V_(in) is sequentially written in the memory 16B of theimage converter 16, and by reading out that source video signal V_(in)on the basis of read address on which image transformation shown by thetransformation matrix T₀ will be provided, the source video signalV_(out) which can be inserted to the studio video signal V_(BK) will beformed.

The source video signal V_(in) to be written in the memory 16B and thesource video signal V_(out) to be read out from the memory 16B are bothtwo-dimensional video data and also the memory 16B is a memory to storethe two-dimensional data. Accordingly, in the read address calculationfor use of read operation from the memory 16B, practically the parameterfor calculating the data of three-dimensional space in the z-axisdirection will not be used. Accordingly, of the transformation matrix Tshown in the equation (3), parameters of the third row and the thirdcolumn for calculating the data in the z-axis direction becomeunnecessary.

More specifically, where the transformation matrix having the parameteractually required for the read address calculation to be T₃₃, thetransformation matrix T₃₃ becomes the matrix exclusive of the third rowand the third column of the equation (3) and can be expressed asfollows: ##EQU4##

Then, the read address calculation method to be used for the readoperation of the source video signal from the memory 16B will beexplained. At first, referring to FIG. 11, the relation between theposition vector on the memory 16B and the position vector on the monitorscreen will be explained. For the sake of clarity, we will deal thematter supposing that two field memories in the memory 16B which are inthe readable state are as one field memory.

Firstly, let the two-dimensional address on the frame memory to be(X_(M), Y_(M)) and the position vector to be [X_(M) Y_(M) ], address onthe monitor screen to be (X_(S), Y_(S)) and the position vector to be[X_(S) Y_(S) ]. Then, if this two-dimensional position vector [X_(M)Y_(M) ] on the frame memory is expressed by the homogeneous coordinate,it can be expressed as vector [x_(m) y_(m) H₀ ] and the position vector[X_(S) Y_(S) ] on the monitor screen can be expressed as vector [x_(s)y_(s) 1]. The parameter "H₀ " of this homogeneous coordinate system isthe parameter to show the magnitude of the vector.

By applying the transformation matrix T₃₃ to the position vector [x_(m)y_(m) H₀ ] on the frame memory, the position vector [x_(m) y_(m) H₀ ]will be transformed to the position vector [x_(s) y_(s) 1] on themonitor screen. Thus, the relationship between the position vector[x_(m) y_(m) H₀ ] on the frame memory and the position vector [x_(s)y_(s) 1] on the monitor screen can be expressed by the followingequation:

    [x.sub.s y.sub.s 1]=[x.sub.m y.sub.m H.sub.0 ]·T.sub.33(5)

The relation between the parameter "H₀ " of the homogeneous coordinateused in the position vector [x_(m) y_(m) H₀ ] on the frame memory andthe parameter "1" of the homogeneous coordinate system used in theposition vector [x_(s) y_(s) 1] on the monitor screen is that theposition vector [x_(m) y_(m) ] on the frame memory is transformed to theposition vector [x_(s) y_(s) ] on the screen by the transformationmatrix T₃₃ and the magnitude "H₀ " of the position vector [x_(m) y_(m) ]on the frame memory is transformed so that the magnitude of the positionvector [x_(s) y_(m) ] on the monitor screen becomes "1".

In the device like this video signal processing device 1 which providesthe spatial transformation processing to the source video signal V_(in)by supplying the read address corresponding to the transformation matrixT₃₃ to the frame memory, it is necessary to obtain the point on theframe memory corresponding to the point on the monitor screen, notobtaining the point on the monitor screen corresponding to the point onthe frame memory. More specifically, as shown in the following equationtransformed from the equation (5), the position vectors [x_(m) y_(m) H₀] on the frame memory should be calculated using the inverse matrix T₃₃⁻¹ of the transformation matrix T₃₃ with respect to the position vectors[x_(s) y_(s) 1] on the monitor screen.

    [x.sub.m y.sub.m H.sub.0 ]=[x.sub.s y.sub.s 1]·T.sub.33.sup.-1(6)

Then, based on this principle, the method actually to obtain the 2-Dposition vectors [X_(M) Y_(M) ] on the frame memory will be explainedbelow. Firstly, the transformation matrix T₃₃ is expressed by parametersa₁₁ -a₃₃ as shown in the following equation: ##EQU5##

And the inverse matrix T₃₃ ⁻¹ will be expressed by the parameters b₁₁-b₃₃ as shown in the following equation: ##EQU6## provided that,##EQU7##

The inverse matrix T₃₃ ⁻¹ thus defined will be substituted into theequation (6) described above and by expanding this, the followingequation will be obtained. ##EQU8##

From this equation (9), the position vectors [x_(m) y_(m) H₀ ] on theframe memory can be expressed as follows:

    x.sub.m =b.sub.11 x.sub.s +b.sub.21 y.sub.s +b.sub.31

    y.sub.m =b.sub.12 x.sub.s +b.sub.22 y.sub.s +b.sub.32

    H.sub.0 =b.sub.13 x.sub.s +b.sub.23 y.sub.s +b.sub.33      (10)

In the case of transforming the homogeneous coordinate position vectors[x_(m) y_(m) H₀ ] on the frame memory thus obtained to the 2-D positionvectors [X_(M) Y_(M) ] on the frame memory, the following procedure willbe recommended. More specifically, since the parameter "H₀ " used whentransforming the 2-D position vectors [X_(M) Y_(M) ] to the homogeneouscoordinate system is the parameter to show the magnitude of the positionvectors [x_(m) y_(m) ] of the homogeneous coordinate system, in order totransform the position vector of the homogeneous coordinate system tothe 2-D position vector, parameters "x_(m) " and "y_(m) " to show thedirection of the position vector of the homogeneous coordinate systemmay be normalized by the parameter "H₀ " to show the magnitude of thehomogeneous coordinate position vector. Thus, each parameter "X_(M) "and "Y_(M) " of the 2-D position vector on the frame memory can beobtained by the following equation:

    X.sub.M =x.sub.m /H.sub.0

    Y.sub.M =y.sub.m /H.sub.0                                  (11)

In the case of transforming the homogeneous position vectors [x_(s)y_(s) 1] on the monitor screen to the 2-D position vectors [X_(S) Y_(S)], the same theorem can be applied and parameters "x_(s) " and "y_(s) "showing the direction of the homogeneous coordinate position vector maybe normalized by the parameter "1" showing the magnitude of thehomogeneous coordinate position vector. Thus, each parameter "x_(s) "and "y_(s) " of the 2-D position vector on the monitor screen can beobtained by the following equation:

    X.sub.S =x.sub.s

    Y.sub.S =y.sub.s                                           (12)

Thus, by substituting the equations (10) and (12) into the equation(11), each parameter "X_(M) " and "Y_(M) " of the 2-D position vector onthe frame memory can be expressed as shown in the following equations:##EQU9##

And as well as the position vectors [X_(M) Y_(M) ] on the frame memorycan be obtained by these equations (13) and (14), the read address(X_(M), Y_(M)) on the frame memory can be obtained.

Then, each parameter of the inverse matrix T₃₃ ⁻¹ to be used in theequations (13) and (14) will be obtained. In utilizing each parametera₁₁ -a₃₃ of the transformation matrix T₃₃, each parameter b₁₁ -b₃₃ ofthe inverse matrix T₃₃ ⁻¹ can be expressed as shown in the followingequations: ##EQU10##

Provided that the parameter W₁ is the value shown in the followingequation:

    W.sub.1 =-a.sub.22 a.sub.31 a.sub.13 +a.sub.21 a.sub.32 a.sub.13 +a.sub.12 a.sub.31 a.sub.23 -a.sub.11 a.sub.32 a.sub.23 -a.sub.12 a.sub.21 a.sub.33 +a.sub.11 a.sub.22 a.sub.33                               (24)

Here, the value of each parameter a₁₁ -a₃₃ will be expressed by thefollowing equations from the equation (7).

    a.sub.11 =r.sub.11, a.sub.12 =r.sub.12, a.sub.13 =r.sub.13 P.sub.z(25)

    a.sub.21 =r.sub.21, a.sub.22 =r.sub.22, a.sub.23 =r.sub.23 P.sub.z(26)

    a.sub.31 =1.sub.x, a.sub.32 =1.sub.y, a.sub.33 =1.sub.z P.sub.z +s(27)

By substituting these equations (25)-(27) into the equations (15)-(24),the equations (15)-(24) can be transformed to the following equations:##EQU11##

    W.sub.1 =-r.sub.22 1.sub.x r.sub.13 P.sub.z +r.sub.21 1.sub.y r.sub.13 P.sub.z +r.sub.12 1.sub.x r.sub.23 P.sub.z -r.sub.11 1.sub.y r.sub.23 P.sub.z -r.sub.12 r.sub.21 (1.sub.z P.sub.z +s)+r.sub.11 r.sub.22 (1.sub.z P.sub.z +s)                                               (37)

Then, by substituting the equations (28)-(37) into the equations (13)and (14), the read address (X_(M), Y_(M)) of the frame memory can beobtained by the following equations: ##EQU12##

And by substituting the equations (34)-(36) into the equation (10), theparameter H₀ can be expressed by the following equation:

    H.sub.0 =(-r.sub.22 1.sub.x +r.sub.21 1.sub.y)X.sub.S +(r.sub.12 1.sub.x -r.sub.11 1.sub.y)Y.sub.S +(-r.sub.12 r.sub.21 +r.sub.11 r.sub.22)(40)

Thus, the read address (X_(M), Y_(M)) of the frame memory can beexpressed using each parameter (r₁₁ -r₃₃, 1_(x), 1_(y), 1_(z), s andP_(z)) of the transformation matrix T₃₃. Accordingly, if the screenaddress (X_(S), Y_(S)) will be supplied from the equation (38) to theequation (40) per pixel in order to correspond in the raster scanningorder of the monitor screen, the read address (X_(M), Y_(M)) on theframe memory corresponding to the screen address supplied can besequentially obtained.

(3-3) Calculation Method of Transformation Matrix T₃₃

As described above, if each parameter of the transformation Matrix T₃₃can be found, the read address (X_(M), Y_(M)) of the frame memory can beobtained utilizing the equations (38)-(40). At this point, thecalculation method of each parameter of this transformation matrix T₃₃will be explained.

The position vector on the frame memory and the position vector on themonitor screen are related as shown in the equation (5) as describedabove. Accordingly, by substituting the actual value of the positionvector into this equation (5), each parameter of the transformationmatrix T₃₃ can be obtained.

As the position vector on the monitor screen, the position vector of 4corners A, B, C, D of the quadrangle ABCD detected by the cornerdetector 9 will be used. Moreover, as the position vector on the framememory, the position vector of 4 corners I, J, K, L of the selectedimage area IJKL specified by the operator will be used. As describedabove, since the address signal S2 on the 4 corner display coordinatesof the selected image area IJKL will be sent out from the controller 18,the position vector on the memory of 4 corners of the selected imagearea IJKL will be calculated on the basis of that address signal S2 andthis will be used as the position vector of corners I, J, K, L.

Firstly, let the position vectors of 4 corners A, B, C, D of thequadrangle ABCD to be [X₁ Y₁ ], [X₂ Y₂ ], [X₃ Y₃ ], [X₄ Y₄ ] in orderand the position vectors of 4 corners I, J, K, L of the selected imagearea IJKL specified by the operator to be [X'₁ Y'₁ ], [X'₂ Y'₂ ], [X'₃Y'₃ ], [X'₄ Y'₄ ]. And as these position vectors are expressed by thehomogeneous coordinate system respectively, the position vectors of 4corners A, B, C, D can be expressed by the following equation:

    [X.sub.i ·K.sub.i Y.sub.i ·K.sub.i K.sub.i ] i=1-4(41)

And the position vectors of 4 corners I, J, K, L can be expressed by thefollowing equation:

    [X'.sub.i Y'.sub.i 1] i=1-4                                (42)

By substituting the position vectors of the homogeneous coordinatesystem into the equation (5) respectively, the following equation willbe obtained:

    [X.sub.i ·K.sub.i Y.sub.i ·K.sub.i K.sub.i ]=[X'.sub.i Y'.sub.i 1]·T.sub.33                             (43)

Here, the transformation matrix T₃₃ is defined as shown in the followingequation: ##EQU13##

And the equation (43) can be transformed as shown in the followingequation: ##EQU14##

And by expanding this equation (45), the following equation will beobtained: ##EQU15##

And regarding each parameter "X_(i) ", "Y_(i) " and "K_(i) ", thefollowing equations can be obtained:

    X.sub.i ·K.sub.i =a.sub.11 X'.sub.i +a.sub.21 Y'.sub.i +a.sub.31(47)

    Y.sub.i ·K.sub.i =a.sub.12 X'.sub.i +a.sub.22 Y'.sub.i +a.sub.32(48)

    K.sub.i =a.sub.13 X'.sub.i +a.sub.23 Y'.sub.i +a.sub.33    (49)

By substituting the equation (49) into the equations (47) and (48),equations on the parameters "X_(i) " and "Y_(i) " will be obtained asfollows: ##EQU16##

At this point, dividing denominators and numerators of the right side ofthese equations (50) and (51) by the parameter "a₃₃ ", these equationsbecome as follows: ##EQU17##

It is clear from the above equations that the values of parameters"X_(i) " and "Y_(i) " do not change if divided by the parameter "a₃₃ ".Accordingly, even though the transformation matrix T₃₃ is replaced withthe transformation matrix T₃₃ ' to be shown in the following equation,the equation (45) exists. ##EQU18##

That is, the following equation exists.

    [X.sub.i ·K.sub.i Y.sub.i ·K.sub.i K.sub.i ]=[X'.sub.i Y'.sub.i 1]·T.sub.33 ' ##EQU19##

Expanding this equation (55) re i=1-4, 12 linear equations relating to"a₁₁ '"-"a₃₃ '" and "K₁ "-"K₄ " as shown in the following equations canbe obtained.

    X.sub.1 ·K.sub.1 =a.sub.11 'X'.sub.1 +a.sub.21 'Y'.sub.1 +a.sub.31 '                                                         (56)

    Y.sub.1 ·K.sub.1 =a.sub.12 'X'.sub.1 +a.sub.22 'Y'.sub.1 +a.sub.32 '                                                         (57)

    K.sub.1 =a.sub.13 'X'.sub.1 +a.sub.23 'Y'.sub.1 +1         (58)

    X.sub.2 ·K.sub.2 =a.sub.11 'X'.sub.2 +a.sub.21 'Y'.sub.2 +a.sub.31 '                                                         (59)

    Y.sub.2 ·K.sub.2 =a.sub.12 'X'.sub.2 +a.sub.22 'Y'.sub.2 +a.sub.32 '                                                         (60)

    K.sub.2 =a.sub.13 'X'.sub.2 +a.sub.23 'Y'.sub.2 +1         (61)

    X.sub.3 ·K.sub.3 =a.sub.11 'X'.sub.3 +a.sub.21 'Y'.sub.3 +a.sub.31 '                                                         (62)

    Y.sub.3 ·K.sub.3 =a.sub.12 'X'.sub.3 +a.sub.22 'Y'.sub.3 +a.sub.32 '                                                         (63)

    K.sub.3 =a.sub.13 'X'.sub.3 +a.sub.23 'Y'.sub.3 +1         (64)

    X.sub.4 ·K.sub.4 =a.sub.11 'X'.sub.4 +a.sub.21 'Y'.sub.4 +a.sub.31 '                                                         (65)

    Y.sub.4 ·K.sub.4 =a.sub.12 'X'.sub.4 +a.sub.22 'Y'.sub.4 +a.sub.32 '                                                         (66)

    K.sub.4 =a.sub.13 'X'.sub.4 +a.sub.23 'Y'.sub.4 +1         (67)

Since this linear equation has 12 parameters, it can be solved.Accordingly, parameters "a₁₁ '"-"a₃₃ '" can be obtained and thetransformation matrix T₃₃ ' can be obtained. In this connection, thetransformation matrix T₃₃ ' can be obtained by multiplying by theparameter "a₃₃ " for scaling up and down to be preset to thetransformation matrix T₃₃ ' obtained.

(3-4) Generation Method of Conversion Address

The three-dimensional conversion address generator 11 forms conversionaddress to supply to the image converter 16 according to the proceduredescribed above. More specifically, the 3-dimensional conversion addressgenerator 11 sets the linear equation on each parameter of thetransformation matrix T₃₃ described above based on the position vectorof 4 corners of the quadrangle ABCD to be supplied as address signal S1from the corner detector 9 and the position vector of 4 corners of theselected image area IJKL to be supplied as address signal S2 from thecontroller 18, and by solving that linear equation, obtains thetransformation matrix T₃₃. Then the 3-dimensional conversion addressgenerator 11 obtains the inverse matrix T₃₃ ⁻¹ using each parameter ofthe transformation matrix T₃₃ obtained, and obtains conversion address(X_(M), Y_(M)) to be supplied to the image converter 16 based on eachparameter of the inverse matrix T₃₃ ⁻¹ and screen address (X_(S), Y_(S))to be supplied from the screen address generator 22 as screen addresssignal S3, and supplies this conversion address to the image converter20 as conversion address signal S4. More specifically, in practice, theprocedure to obtain the inverse matrix T₃₃ ⁻¹ from the transformationmatrix T₃₃ is omitted and instead, calculations of the equations(38)-(40) described above are conducted by utilizing each parameter ofthe transformation matrix T₃₃, and the conversion address (X_(M), Y_(M))will be obtained directly.

(4) Operation and Effects of the Embodiment

According to the foregoing construction, in this video signal processingdevice 1, a blue board 6 is set for as a target of insertion of thesource video image in the background of a newscaster 4, and this blueboard 6 is photographed with the newscaster 4 by the video camera 2 andstudio video signal V_(BK) is formed. This studio video signal V_(BK) issupplied to the chroma key device and target key signal keyT showing thearea having blue hue is formed. The corner detector 9, receiving thistarget key signal keyT, detects positions of 4 corners A, B, C, D of thequadrangle ABCD shown by the blue board 6 based on the target key signalkeyT, and supplies the address signal S1 showing that positions to the3-dimensional conversion address generator 11.

On the other hand, the source video signal V_(in) which is formedreproduced by the video tape recorder 12 or photographed by the videocamera 13 is supplied to the image converter 16 and sequentially writtenin the memory 16B provided in the image converter 16. Moreover, thesource video signal V_(in) is also supplied to the monitor 17 anddisplayed on this monitor 17. The operator who operates the video signalprocessing device 1 operates the input device 10 observing the sourcevideo signal V_(in) to be displayed on this monitor 17 and specifies therange of selected image area IJKL to fit to the quadrangle ABCD of thestudio video signal V_(BK). This range specification information will besent out to the controller 18 from the input device 10. The controller18 detects positions of 4 corners I, J, K, L of the selected image areaIJKL based on the range specification information and supplies theaddress signal S2 showing that positions to the 3-dimensional conversionaddress generator 11 and the self key generator 19.

The 3-dimensional conversion address generator 11 calculates conversionaddress for image converting the selected image area IJKL to the sameshape as the quadrangle ABCD based on the address signal S1 showing thepositions of 4 corners of the quadrangle ABCD supplied from the cornerdetector 9 and the address signal S2 showing the positions of 4 cornersof the selected image area IJKL supplied from the controller 18. In thecase of obtaining the conversion address, the 3-dimensional conversionaddress generator 11 firstly obtains the transformation matrix T₃₃ ofthe 3-dimensional image conversion processing based on the positionvectors of 4 corners of the quadrangle ABCD and the position vectors of4 corners of the selected image area IJKL. Then, the 3-dimensionaladdress generator 11 obtains the inverse matrix T₃₃ ⁻¹ of thetransformation matrix T₃₃ using each parameter of that transformationmatrix T₃₃, and by conducting the calculation processing based on eachparameter of this inverse matrix T₃₃ ⁻¹ and screen address (X_(s),Y_(s)) from the screen address generator 22, obtains conversion addresssequentially and supplies this to the image converter 16 and thetransducer 20 as conversion address signal S4.

The image converter 16 sequentially reads out the source video signalV_(in) written in the memory 16B based on the conversion address signalS4. Thus, the source video signal V_(out) which is 3-dimensional imageconversion processed so that it can be inserted to the quadrangle ABCDof the studio video signal V_(BK) will be formed.

Furthermore, the self key generator 19 receives the address signal S2showing the positions of 4 corners I, J, K, L of the selected image areaIJKL from the controller 18, and based on said address signal S2, itforms self key signal S5 in which the area corresponding to the shape ofthe selected image area IJKL is formed with the signal level "1" and theother areas are formed with the signal level "0". The transducer 20writes this self key signal S5 in the memory and reads this out based onthe conversion address signal S4 supplied from the 3-dimensionalconversion address generator 11. Accordingly, the self key signalK_(out) in which the area having the signal level "1" is transformed tothe same shape as the quadrangle ABCD will be formed.

The mixer 8, upon switching the image converted source video signalV_(out) and the studio video signal V_(BK), sends it out. Morespecifically, when the signal level of the self key signal K_(out) is"0", the studio video signal V_(BK) is selected and sent out, and whenthe signal level of the self key signal K_(out) is "1", the source videosignal V_(out) is selected and sent out. And thus, the composite videosignal V_(mix) in which the source video signal V_(out) is inserted tothe quadrangle ABCD of the studio video signal V_(BK) will be formed.

At this point images of each video signal will be shown in FIGS. 12 and13. As shown in FIGS. 12A to 12C, of the source video signal V_(in), thepart specified as the selected image area IJKL will be image transformedbased on the target key signal keyT showing the shape of the quadrangleABCD and will be transformed to the shape of the quadrangle ABCD asshown in FIG. 12C. This transformed source video signal V_(out) iscombined to the quadrangle ABCD of the studio video signal V_(BK) asshown in FIGS. 13A to 13C, and as a result, the composite video signalV_(mix) in which the selected image area IJKL is inserted to thequadrangle ABCD will be formed.

With this arrangement, in this video signal processing device 1, thepositions of 4 corners of the quadrangle ABCD is detected from thestudio video signal V_(BK), the transformation matrix T₃₃ fortransforming the image is calculated based on the position informationshowing the position detected and the position information to show the4-corner positions of the selected image area IJKL to be inserted, andby using each parameter of that transformation matrix T₃₃, the inversematrix T₃₃ ⁻¹ of the transformation matrix T₃₃ is obtained, and based oneach parameter of that inverse matrix T₃₃ ⁻¹ and the screen address,conversion address for image transformation is calculated, and thesource video signal V_(in) written in the memory 16B of the imageconverter 16 is read out based on that conversion address. Thus, thesource video signal V_(out) having the source video image that fits tothe prescribed frame (ABCD) in the background of the newscaster 4 can beautomatically formed without the operator's input of the parametershowing the shape after converted using such as the trackball as theconventional device. Accordingly, the operator has to conduct verysimply operation just to specify the selected image area IJKL to beinserted to the studio video signal V_(BK) (in case of inserting theoverall source video signal V_(in), this operation is not required), andthe complicated manual adjustment to fit the source video imagecorrectly to the prescribed frame (ABCD) as before becomes unnecessaryand thus, manipulation of the operator can be further decreased thanbefore.

According to the foregoing construction, since detecting 4 cornerpositions of the quadrangle ABCD into which the source video image isinserted from the studio video signal V_(BK), calculating thetransformation matrix T₃₃ for image conversion based on the positioninformation to show the positions detected and the position informationto show 4-corner positions of the selected image area IJKL to beinserted, by using each parameter of that transformation matrix T₃₃, theinverse matrix T₃₃ ⁻¹ of the transformation matrix T₃₃ is calculated,the conversion address for image conversion is calculated based on eachparameter of that inverse matrix T₃₃ ⁻¹ and the screen address, andbased on that conversion address, source video signal V_(in) will beread out from the memory 16B, the source video signal V_(out) whichcorrectly fits to the quadrangle ABCD of the studio video signal V_(BK)can be automatically formed. Thus, in the case of inserting the sourcevideo image into the prescribed frame ABCD in the background of thenewscaster 4, the source video image can be automatically inserted tothe blue board area without the operator's entering the parameter toshow the condition after conversion. Accordingly, the operator'smanipulation can be further decreased as compared with the conventionaldevice and the operability can be further improved.

(5) Other Embodiments

(5-1) The embodiment described above has dealt with the case ofdisplaying the source video signal V_(in) on the monitor 17 andspecifying the selected image area IJKL by observing the screen of thesource video signal V_(in) to be shown on this monitor 17. However, thepresent invention is not only limited to this but also providing onlythe monitor 21 on which the composite video signal V_(mix) is displayedwithout providing the monitor 17, and if the selected image area IJKLwould be specified by observing the screen of composite video signalV_(mix) to be displayed on the monitor 21, the same effects as those ofthe embodiment described above can be obtained.

This specification method of the selected image area IJKL by observingthe monitor 21 will be explained more specifically in the followingparagraphs. The range specification information of the selected imagearea IJKL is set plus 100 percent and minus 100 percent in thehorizontal direction from the center of the effective image area EFGH,and plus 100 percent and minus 100 percent in the vertical direction asthe default value, and the whole area of the effective image area EFGHis specified as the selected image area IJKL. Accordingly, immediatelyafter the video signal processing device 1 is started operating, thecomposite video signal V_(mix) in which the effective image area EFGH isinserted into the prescribed frame ABCD in the background of thenewscaster 4 will be displayed on the monitor 21.

The operator, observing the screen of the composite video signal V_(mix)to be displayed on this monitor 21, operates the range specificationvolume and the keyboard and enters the range specification informationfor specifying the selected image area IJKL. For example, when the rangespecification information of the selected image area IJKL issequentially converted by operating the range specification volume, therange of the selected image area IJKL displayed on the monitor 21changes sequentially. The operator observes the change of the range ofthis selected image area IJKL, and when the desired range of theselected image area IJKL is displayed, stops input operation and fixesthe range specification volume. Accordingly, the desired selective imagearea IJKL can be inserted to the prescribed frame ABCD in the backgroundof the newscaster.

Furthermore, it is possible to input the value of the rangespecification information directly by using the keyboard instead of therange specification volume. In this case, for example, as the rangespecification information in the horizontal direction, if plus 80percent and minus 80 percent from the center of the effective image areaEFGH are put in, and as the range specification information in thevertical direction, if plus 80 percent and minus 80 percent from thecenter of the effective image area EFGH are put in, that range in theeffective image area EFGH will be selected as the selected image areaIJKL. Accordingly, the composite video signal V_(mix) in which thisselected image area IJKL is inserted into the prescribed frame ABCD inthe background of the newscaster 4 is displayed on the monitor 21.

In this connection, in the case of conducting these rangespecifications, operation of the controller 18 is basically the sameexcept there is no display control to the monitor 17. More specifically,the controller 18 detects positions of 4 corners of the selective imagearea IJKL based on the range specification information received from theinput device 10 and outputs the address signal S2 to show thatpositions.

(5-2) Furthermore, as shown in FIG. 14, the embodiment described abovehas dealt with the case of inserting the selected image area IJKL to thequadrangle ABCD in order that corners I, J, K, L of the selected imagearea IJKL correspond to corners A, B, C, D of the quadrangle ABCDrespectively. However, the present invention is not only limited to thisbut also by shifting this correlation by 90-degree, the selected imagearea IJKL may be inserted changing its direction.

This insertion method after changing direction of the selected imagearea IJKL will be described in detail as follows: The operator entersdirection information to show the relation between each corner of theselective image area IJKL and each corner of the quadrangle ABCD withthe range specification information to specify the selected image areaIJKL through the input device 10. As to this direction information, theangle of rotation to make the corners to correspond will be entered byrotating the selected image area IJKL. Let the clockwise direction ofthe rotation angle to be plus direction and the counter-clockwisedirection to be minus direction.

For example, if plus 90-degree is entered as the direction informationtogether with the range specification information from the input device10, the controller 18 receives these information. And the controller 18detects 4 corner positions of the selected image area IJKL based on therange specification information and sends out the direction informationwith the address signal S2 showing that positions to the 3-dimensionalconversion address generator 11.

The 3-dimensional conversion address generator 11 generates theconversion address which makes the selective image area IJKL to fit intothe quadrangle BDAC (i.e., the conversion address which makes theselected image area IJKL to make a 90-degree turn to correspond) by anarithmetic operation based on the address signal S1 to show positions of4 corners of the quadrangle ABCD supplied from the corner detector 9,the address signal S2 to show 4 corner positions of the selected imagearea IJKL supplied from the controller 18 and the direction informationto show plus 90-degree, as shown in FIG. 15. Thus, by supplying thisconversion address to the image converter 16 and the transducer 20 forkey signal, the composite video signal V_(mix) in which the selectedimage area IJKL is inserted in a state of 90-degree turn in theclockwise will be formed.

Furthermore, in the case where the direction information showing plus180-degree is entered as the direction information from the input device10, the controller 18 supplies this direction information showing plus180-degree to the 3-dimensional conversion address generator 11 with theaddress signal S2 showing positions of 4 corners of the selected imagearea IJKL. The 3-dimensional conversion address generator 11 formsconversion address so that the selected image area IJKL fits to thequadrangle DCBA based on the address signal S1 to show the positions of4 corners of the quadrangle ABCD supplied from the corner detector 9,the address signal S2 to show positions of 4 corners of the selectedimage area IJKL supplied from the controller 18 and the directioninformation to show plus 180-degree by an arithmetic operation as shownin FIG. 16 (i.e., the conversion address that makes the selective imagearea IJKL to make 180-degree turn to correspond). Thus, by supplyingthis conversion address to the image converter 16 and the transducer 20for key signal, the composite video signal V_(mix) in which theselective image area IJKL is inserted in a state of 90-degree turn inthe clockwise direction will be formed.

Furthermore, in the case where the direction information showing plus270-degree is entered as the direction information from the input device10, the controller 18 supplies this direction information showing plus270-degree together with the address signal S2 showing the positions of4 corners of the selected image area IJKL to the 3-dimensionalconversion address generator 11. The 3-dimensional conversion addressgenerator 11, as shown in FIG. 17, forms the conversion address (i.e.,the conversion address that makes the selective image area IJKL to make270-degree turn to correspond) to fit into the quadrangle CADB of theselective image area IJKL by an arithmetic operation. Thus, by supplyingthis conversion address to the image converter 16 and the transducer 20for key signal, the composite video signal V_(mix) in which theselective image area IJKL is inserted in a state of 270-degree turn inthe clockwise can be formed.

(5-3) Moreover, the embodiment described above has dealt with the caseof specifying rectangle or square selected image area IJKL by supplyingthe range specification information of horizontal and verticaldirection. However, the present invention is not only limited to thisbut also the position of each corner I, J, K, L of the selected imagearea IJKL may be selected by using the input device 10, such askeyboard. With this arrangement, as shown in FIGS. 18A to 18C, theselected image area IJKL having an optional shape not simple rectangleor square form can be inserted to the quadrangle ABCD, thus theoperability can be further improved.

(5-4) Furthermore, the embodiment described above has dealt with thecase of generating the key signal (S5) to show the shape of source videoimage inside the video signal processing device 1. However, the presentinvention is not only limited to this but also key signal may bereceived from the external equipment. The construction of the videosignal processing device 1 according to this case will be shown in FIG.19, in which the corresponding parts of FIG. 1 are designated the samereference numerals.

In the case of this video signal processing device 30, the source videosignal V_(in) ' in which the other image processing was provided by theexternal equipment (not shown in Figures) will be entered. This sourcevideo signal V_(in) ' is supplied to the image converter 16 as in thecase of video signal processing device 1 shown in FIG. 1 andsuccessively written in the memory in the image converter 16. Moreover,in the case of this video signal processing device 30, key signal keySformed in the external equipment with the source video signal V_(in) 'is entered. This key signal keyS is the signal to show the shape of anarea to be inserted to the quadrangle ABCD as the source video imagefrom among the source video signal V_(in) ', and the signal levelbecomes "1" in the area corresponding to the image area to be insertedand the signal level becomes "0" outside that area. This key signal keySwill be entered to the corner detector 31 and the transducer 20.

The corner detector 31 has the similar construction to that of thecorner detector 9 to detect the corner of target key signal keyT anddetects 4 corner positions of the key signal keyS, and supplies addresssignal S20 showing the 4 corner positions in the display coordinates tothe 3-dimensional conversion address generator 11.

The 3-dimensional conversion address converter 11 calculates thetransformation matrix to insert the source video signal V_(in) ' to thequadrangle ABCD based on the address signal S1 showing 4 cornerpositions of the quadrangle ABCD supplied from the corner detector 9 andthe address signal S20 showing 4 corner positions of the key signal keySsupplied from the corner detector 31, and calculates the conversionaddress based on the inverse matrix of that transformation matrix andthe screen address signal S3 from the screen address generator 22. Morespecifically, in the case of this video signal processing device 30, theconversion address will be obtained by using the position information of4 corners of key signal keyS detected at the corner detector 31 in placeof the position information of 4 corners of the selective image areaIJKL.

The conversion address obtained will be supplied to the image converter16 and the transducer 20 for key signal as conversion address signal S4.The image converter 16 forms source video signal V_(out) imagetransformed by reading out the source video signal V_(in) written in theinside memory based on the conversion address obtained by the conversionaddress signal S4. Similarly, the transducer 20, by reading out the keysignal keyS written in the inside memory based on the conversion addressto be obtained by the conversion address signal S4, forms key signalK_(out) in which the area its signal level becomes "1" is transformed tothe same shape as the quadrangle ABCD. Thus, in the mixer 8, byoutputting the source video signal V_(out) and the studio video signalV_(BK) selectively based on this key signal K_(out) the composite videosignal V_(mix) in which the source video signal V_(in) ' is insertedwill be formed.

(5-5) Furthermore, the embodiment described above has dealt with thecase where the destination of source video image insertion was aquadrangle ABCD. However, the present invention is not only limited tothis but also the destination of source video image insertion may be anypolygon having more than 4 corners, because if there are more than 4corners at least, each parameter of the transformation matrix T₃₃ can becalculated.

(5-6) Moreover, according to the embodiment as described above, in thecase of not inserting the source video signal V_(in), the blue board 6was removed by using an elevator. However, the present invention is notonly limited to this but also the studio staff may remove the blue board6.

According to the present invention as described above, since each cornerposition of the blue board area is detected respectively from the videosignal, and the conversion address is generated based on the positioninformation showing the position detected and the position informationshowing each corner position of the image area of the source videosignal to be inserted, and conversion source video signal is formedbased on that conversion address, the image area can be automaticallyinserted into the blue board area, that is a destination of insertion,without the operator entering the parameter to show the condition afterconversion as before, the operation of the operator can be furtherdecreased as compared with the past. Thus, a video signal processingdevice and a video signal processing method capable of furtherdecreasing the operator's operation can be realized.

While there has been described in connection with the preferredembodiments of the invention, it will be obvious to those skilled in theart that various changes and modifications may be aimed, therefore, tocover in the appended claims all such changes and modifications as fallwithin the true spirit and scope of the invention.

What is claimed is:
 1. A video signal processing apparatus for mixing afirst video signal and a second video signal, comprising:means fordetecting a color area of a prescribed color in said first video signal;transforming means for performing a 3-D transformation on said secondvideo signal and on a key signal corresponding to said second videosignal to generate a transformed second video signal and a transformedkey signal; and means for mixing said first video signal and saidtransformed second video signal according to said transformed key signalsuch that said transformed second video signal corresponds to said colorarea of said first video signal.
 2. The video signal processingapparatus according to claim 1, further comprising means for calculatingcoordinates of corners of said color area and coordinates of corners ofan area corresponding to said key signal.
 3. The video signal processingapparatus according to claim 2, wherein said transforming meanscalculates 3-D transformation parameters and said 3-D transformation isbased on said 3-D transformation parameters, said coordinates of cornersof said color area and said coordinates of corners of said areacorresponding to said key signal.
 4. The video signal processingapparatus according to claim 3, wherein said transforming means includesmemory means for storing said second video signal and said key signaland address generation means for generating read addresses based on said3-D transformation parameters, said read addresses being used forreading from said memory means.
 5. A method for mixing a first videosignal and a second video signal, comprising the steps of:detecting acolor area of a prescribed color in said first video signal; performinga 3-D transformation on said second video signal and on a key signalcorresponding to said second video signal to generate a transformedsecond video signal and a transformed key signal; and mixing said firstvideo signal and said transformed second video signal according to saidtransformed key signal such that said transformed second video signalcorresponds to said color area of said first video signal.
 6. The methodaccording to claim 5, further comprising the step of calculatingcoordinates of corners of said color area and coordinates of corners ofan area corresponding to said key signal.
 7. The method according toclaim 6, wherein said step of performing a 3-D transformation includescalculating 3-D transformation parameters, and wherein said 3-Dtransformation is based on said 3-D transformation parameters, saidcoordinates of corners of said color area and said coordinates ofcorners of said area corresponding to said key signal.
 8. The methodaccording to claim 7, wherein said step of performing a 3-Dtransformation includes storing said second video signal and said keysignal in a memory means and generating read addresses based on said 3-Dtransformation parameters, said read addresses being used for readingfrom said memory means.
 9. A video signal processing apparatus formixing a first video signal and a second video signal, comprising:meansfor receiving said first video signal, said second video signal and asecond key signal corresponding to said second video signal; means forgenerating a target signal indicating a prescribed color area in saidfirst video signal; means for calculating a position of said color areaindicated by said target signal and a position of an area indicated bysaid key signal; means for generating 3-D transformation parametersbased on said position of said color area and said position of said areaindicated by said key signal; means for performing a 3-D transformationon said second video signal and on said key signal according to said 3-Dtransformation parameters in order to generate a transformed secondvideo signal and a transformed key signal; and means for mixing saidfirst video signal and said transformed second video signal according tosaid transformed key signal such that said transformed second videosignal corresponds to said color area of said first video signal.
 10. Amethod for mixing a first video signal and a second video signal,comprising the steps of:receiving said first video signal, said secondvideo signal and a key signal corresponding to said second video signal;generating a target signal indicating a prescribed color area in saidfirst video signal; calculating a position of said color area indicatedby said target signal and a position of an area indicated by said keysignal; generating 3-D transformation parameters based on said positionof said color area and said position of said area indicated by said keysignal; performing a 3-D transformation on said second video signal andsaid key signal according to said 3-D transformation parameters in orderto generate a transformed second video signal and a transformed keysignal; and mixing said first video signal and said transformed secondvideo signal according to said transformed key signal such that saidtransformed second video signal corresponds to said color area of saidfirst video signal.